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A MODEL  OF  A MEMORY  MECHANISM 


R.  Granovskaya,  I.  Bereznaya 

For  the  structural  input  and  output  of  the  mode^,  we  use  a 
most  simple  uniform  linear  circuit  made  of  logical  neuron-like 
elements  which  are  Joined  with  one  anotehr  sequentially  and,  by  a 
method  of  processing  the  input  information,  can  be  compared  with 
on-off-neurons  [1]. 

Besides  the  on-off-neurons,  the  structure  of  the  entire  model 
Includes  a number  of  other  elements,  the  mathematical  models  of 
which  are  obtained  from  some  primary  model  by  superimposing 
restrictions  specific  for  each  type  of  elements  on  it. 

As  the  Initial,  we  took  a model  of  an  adding  neuron  of  a 
threshold  element  which  was  detected  by  physiologists  at  all 
levels  of  the  CNS  (central  nervous  system).  This  neuron  possesses 
a memory  at  each  input  in  the  form  of  synaptic  weight . Models  of 
all  other  elements  used  in  the  memory  model  are  constructed  on  the 
basis  of  the  hypothesis  that  all  specific  neurons  in  the  CNS  are 
derivatives  of  the  adding  neuron  and  are  formed  by  limitations 
which  are  connected  with  the  position  of  the  neurons  in  the  CNS 
and  with  the  specific  nature  of  the  external  signals  [2]. 
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The  model  of  the  on-off-neuron  reacts  to  the  threshold 
differential  of  signals  in  time  at  the  inputs.  It  is  compared 
with  the  neurons  which  were  first  detected  in  the  visual  system 
of  frongs,  and  subsequently,  in  the  hearing,  sensory,  and  other 
parts  of  the  CNS. 


A model  of  the  "neuron  of  innovation"  reacts  to  the  threshold 
differential  of  signals  in  its  inputs,  and  the  neuron  of  identity  - 
to  the  constancy  of  signals  in  its  inputs  or  in  time  (the  neuron 
of  identity  in  space  or  time). 

The  model  of  the  "pH  neuron"  reacts  in  the  case  where  there 
is  a simultaneous  threshold  differntial  of  signals  in  the  space 


of  its  inputs  and  in  time;  with  a sufficiently  low  threshold  it 
will  react  only  to  the  threshold  in  time,  and  with  an  increase 
in  threshold  - only  to  the  aggregate  of  these  conditions. 

The  models  of  "neurons  of  identity"  in  time  and  space  react 
only  when  either  all  of  their  inputs  are  in  Identical  states  or 
one  input  maintains  its  state  Invariable  in  time.  This  is 
equivalent  to  the  mathematical  concept  of  identical  functions.  In 
this  work  these  neurons  are  called  neurons  of  innovation,  where 
by  innovation  there  is  meant  the  presence  of  differentials  in 
the  space  of  the  neuron's  inputs. 


Conditions  for  triggering  various  types  of  neurons  have  the 
following  form: 

Adding  neuron  (£): 


p<*  + 0 


ik  » 

2 2 S,(t)e,(t)-m(t]  . 

I- 1 uMt 


A difference  neuron  of  the  on-off  type 
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A "neuron  of  innovation" 


P(t  + 1)-1 


[ 2 


-m(t)  j. 


A "neuron  of  identity  in  space" 

p(t+  1)  = 1 ISre,  — m (t)  ]. 


m(t)«m(0)  + 2 I S. e, (t)  — Sj e, (t) | . 


A "neuron  of  identity  in  time" 


P «+ 1)  — 8 IS,-  e,  — 0,  (t)l,  S,e,  >m(o), 

■ (t)J„(0)+  2-S,|e, 

Wl“ll 

1 (z)  — I i'*  ’a'°  »f,l  M.»>0 
|o.,<o  6(,)  “ jo.KO 


Pt  - the  state  of  the  output  at  moment  t; 
li(t)  - state  of  the  i-th  input; 
m(t)  - threshold  m(o),  threshold  with  t = 0; 
x - internal  delay 

lr  - state  of  the  input  from  the  generating  cell; 
k - number  of  inputs; 

- synaptic  coefficient  of  the  i-th  input; 

Sr  - synaptic  coefficient  of  the  input  from  the  generating  cell, 

The  logical  function  realized  by  the  adding  neurons  is 
determined  by  their  thresholds.  The  on-off  and  "innovation" 
neurons  realize  the  logical  function  of  addition  modulo  2,  and 
the  "neurons  of  identity"  realize  the  logical  functions  of 
equivalence. 

A time-ordered  binary  M-bit  code  combination,  called  the 
M-code  arrives  sequentially,  bit  by  bit,  at  the  input  of  the 
basic  circuit  as  the  information  on  the  objects. 
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To  begin  with,  we  will  examine  a model  of  the  structure 
which  consists  only  of  the  basic  circuit,  the  elements  of  which 
(the  on-off  neurons)  do  not  possess  memory.  In  this  structure 
information  on  the  objects  can  only  be  stored  in  dynamic  form. 

The  specific  nature  of  the  transformation  of  the  binary  M-code 
by  an  on-off  type  neuron  lies  in  the  fact  that  the  reciprocal 
substitution  of  zero  and  one  in  the  input  code  does  not  change  the 
process  of  transformation  in  the  basic  circuit. 

An  analysis  of  the  process  of  transformation  of  the  M-codes 
by  the  basic  circuit  showed  that  cases  where  the  input  code  has 
length  M = 2n  bits  it  has  specific  peculiarities  (length  of  the 
examined  basic  circuit  was  limited  to  20  neurons).  Passing 
sequentially  through  the  neurons  of  the  basic  circuit,  this  code 
first  is  converted  to  a periodic  M/2-bit  code,  then  after  the 
next  several  elements  - to  an  M/4-bit  code,  etc.,  then  to  an 
M/2n-v-bit  code,  and  then,  after  two  neurons,  it  becomes  a zero 
code. 

In  the  case  of  the  transformation  of  the  code  of  bit  con- 
figuration M / 2n,  we  can  discriminate  from  the  basic  circuit  .a 
group  of  series  connected  neurons  in  which  there  develops  a 
periodic  process  of  transformation  of  the  codes  of  the  neurons 
with  a period  0;  here  the  code  does  not  exist  as  zero  at  the 
output  of  any  single  element  of  the  basic  circuit.  The  value  of 
0 depends  on  M and  not  on  the  type  of  code. 

The  transformation  of  codes  in  the  basic  circuit  can  be 
compared  with  an  autonomous  finite  automaton  which,  as  is  known, 
can  either  achieve  equilibrium  for  a finite  number  of  cycles  or 
periodically  repeat  the  finite  sequence  of  states.  The  M-code  can 
be  compared  with  the  information  acquired  with  the  bypassing  of 
some  object's  loop  If  the  loop  is  closed  and  divided  Into  M 
Intervals,  and  each  Interval  is  compared  with  zero  or  with  one. 


If  the  object's  loop  Is  divided  into  M = 2n  intervals  and  if 
the  result  of  the  bypass  of  the  object  over  the  loop  is  fed  to 
the  input  of  the  basic  circuit  from  the  on-off  neurons,  then  a 
zero  code  will  be  at  the  output  of  the  M-th  neuron  of  the  circuit. 

The  described  stages  of  transformation  of  the  M-code  can  be 
compared  with  the  individual  stages  of  analysis  of  the  loop  of 
the  object.  For  storage  and  analysis  of  the  objects  with  a 
quantity  of  intervals  equal  to  2n,  there  is  no  sense  in  circuits 
being  longer  than  the  2n  neurons.  If  it  is  necessary  to  erase 
some  input  code  from  th'1  memory,  then  it  is  sufficient  to  supple- 
ment it  art'  rily  up  to  an  overall  digit  in  the  period  up  to 
2n. 

If  -iic  object  has  M ^ 2n  intervals  and  if  the  result  of 
bypassing  the  object  over  to  the  loop  is  passed  through  the 
circuit  from  the  on-off  neurons  then  the  periodic  process  of 
transformation  along  the  length  of  the  circuit  takes  place.  The 
period  can  be  less  than,  equal  to,  or  greater  than  M depending 
on  the  number  of  M. 

The  result  of  the  bypassing  of  the  object  over  the  loop  of 
the  M-code  has  an  internal  subperiod,  i.e.,  the  object  has 
repetitive  properties  on  the  aggregate  of  a number  of  adjacent 
intervals,  and  if  the  number  of  digits  in  one  subperiod  is  m = 2n, 
then  this  code  is  converted  to  a zero  code  already  at  the  output 
of  the  m-th  neuron  of  the  circuit.  In  other  words,  for  its 
analysis  we  can  us  a circuit  which  is  shorter  than  that  which 
would  be  required  for  an  analysis  of  an  object  without  internal 
periodicity  of  properties.  The  investigation  of  the  transformation 
of  the  input  codes  by  elements  of  the  basic  circuit  showed  that 
we  can  restore  the  input  information  without  its  conversion  to 
static  storage  [3*  **].  In  this  case,  the  number  of  digits  of  the 
restored  input  code  is  limited  by  the  length  of  the  basic  circuit. 
The  restoration  mechanism  in  this  case  is  such  that  from  the 
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information  on  the  state  of  all  the  elements  of  the  basic  circuit 
at  the  recorded  moment  of  time  we  can  restore  the  state  of  the 
circuit  at  all  the  preceding  moments  in  time,  and  also  the  input 
code  - according  to  certain  recurrent  logical  relationships.  It 
has  been  shown  that  the  algorithm  for  restoration  of  the  input 
code  can  be  realized  in  this  manner,  for  example,  with  the  aid  of 
a network  of  models  of  both  innovation  and  adding  neurons. 

As  an  element  of  the  structure  of  a static  memory  we  intro- 
duced variable  synaptic  weights  of  each  neuron  which  reflected 
the  state  of  a neuron  of  the  basic  circuit  with  the  arrival  of 
an  M-code  at  its  input.  The  synaptic  coefficient  of  the  input 
of  a neuron  changes  by  a fixed  value  depending  on  the  code 
situation  at  the  input  of  this  neuron.  By  the  code  situation  at 
the  input  we  mean  a pair  of  binary  symbols  which  corresponds  to 
the  values  of  the  adjacent  bits  of  the  M-code  at  the  adjacent 
moments  of  time.  We  used  a system  of  three  synaptic  coefficients 
which  correspond  to  the  storage  of:  1)  the  input  situation  00  - 

the  increase  of  the  synaptic  coefficient  in  this  case  is  indicated 
by  o,  2)  the  input  situation  01  or  10  - 0,  3)  input  situation  11  - 
y.  Each  of  these  three  synaptic  coefficients  of  the  neuron  is 
greater  than  zero  by  as  many  whole  units  as  the  number  of  times 
that  at  adjacent  moments  of  time  situations  00,  01,  or  10  and  11 
were  encountered  at  the  input  of  the  neuron. 

For  a transfer  of  Information  about  the  input  code  to  static 
storage,  elements  are  necessary  which  react  only  to  situations 
a,  $ and  y.  We  used  as  such  elements  models  of  neurons  of 
identity  in  time  Tg  and  on-off  as  these  elements,  the  set  of  which, 
together  with  the  adding  neurons,  is  also  the  static  memory  of 
each  element  of  the  basic  circuit. 

Thus,  the  time-ordered  binary  code  sequence  at  the  input 
of  the  circuit  of  the  on-off  neurons  and  each  neuron  separately 
changes  with  the  ordering  of  values  of  synaptic  weights  in  the 
space  of  the  memory  circuit. 
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in  connection  with  the  property  of  the  used  model  of  the 
on-off  neuron  to  react  identically  to  switching  on  and  off,  some 
M-codes  for  it  will  be  equivalent,  i.e.,  passing  through  such  a 
neuron,  they  will  leave  identical  traces  in  its  memory  - the 
coefficients  a,  3,  and  y will  acquire  identical  increases. 
Specifically,  codes  of  one  class  of  equivalence  are  equivalent 
for  the  on-off  neuron.  The  number  of  classes  of  equivalence  for 
various  M is  determined: 


where 


:H(m)  = 

v(d.M)  = 


-jjf-  2 (2d  — 2)  v (d,M), 

If)  , 

(\  ecM  d = 1 

1,  eciu^d  = M 

— 1,  edit?  d = M,  1 


(1)  if 


with  a the  internal  subperiod  in  the  input  code. 


It  has  been  shown  that  according  to  values  a,  3,  and  y of 
one  neuron  of  the  basic  circuit  we  can  restore  the  code  which 
passed  through  it  with  some  accuracy  [5].  In  a number  of  special 
cases  this  accuracy  is  determined  by  one  class  of  equivalence.  In 
these  cases  we  can,  with  an  accuracy  up  to  the  equivalence  class, 
restore  the  input  M-code  for  the  entire  basic  circuit  if,  besides 
a,  3,  and  y of  one  i-th  neuron,  one  of  the  two  symbols  a and  y of 
the  preceding  neurons  is  determined. 

In  the  proposed  model  of  memory,  the  length  of  the  restored 
input  code  does  not  depend  on  the  length  of  the  basic  circuit, 
but  is  limited  only  by  the  range  of  changes  of  o,  3,  and  y. 

The  model  of  static  memory  includes  the  basic  circuit  of 
on-off  neurons  with  a variable  threshold  (which  permits  separating 
the  processes  of  Instruction  and  reading).  Connected  with  each 
of  the  elements  of  the  basic  circuit  is  a network  of  its  static 
memory  which  in  turn  consists  of  neurons  which  discriminate 
situations  a,  3,  and  y,  and  adding  neurons  (counters)  with  a 


PTD-ID(RS) 1-0238-77 


7 


0 

variable  threshold,  and  also  its  restoration  circuit.  This 
network  is  connected  with  counters  and  permits  comparing  their 
contents  with  the  corresponding  parameters  of  the  examination 
input  code. 

The  proposed  model  of  static  memory  operates  in  an  identi- 
fication mode:  if  in  the  reading  mode  at  the  input  of  the  model 

there  arrives  an  examination  code  which  has  a similarity  for  the 
assigned  number  of  symbols  to  the  code  which  passed  through  the 
basic  circuit  in  the  recording  mode,  then  a signal  of  coincidence 
appears  at  the  output  of  the  model. 

The  process  of  restoration  is  described  in  the  form  of  a 
recurrent  algorithm.  In  each  part  of  this  algorithm  the  presented 
code  is  compared  with  the  information  in  the  counters  of  one 
neuron  of  the  basic  circuit;  in  the  case  of  detection  of  a 
coincidence  with  regrd  to  symbols  a,  3,  and  y the  code  is  trans- 
formed according  to  a specific  rule  to  another  code  which  then 
is  analyzed  at  the  next  step. 

The  similarity  of  the  examination  code  to  the  stored  code 
is  manifested  within  the  limits  of  the  memory  of  one  neuron  up 
to  the  memory  of  all  on-off  neurons  of  the  basic  circuit,  i.e., 
the  greater  the  part  of  the  algorithm  accomplished  by  the  given 
examination  code,  the  greater  its  similarity  to  the  code  stored 
in  the  memory.  Thus,  the  described  network  also  provides  the 
possibility  of  introducing  a quantitative  estimate  of  the  similar- 
ity from  the  degree  of  utilization  of  the  algorithm  by  the  exam- 
ination code. 

A special  feature  of  the  described  network  is  the  fact  that 
information  on  the  object  is  not  connected  with  the  specific 
nature  of  a concrete  problem  of  classification,  and  therefore 
the  stored  information  on  the  object  can  also  be  used  for  solving 
problems  of  classification  according  to  any  criteria,  and  for 
identifying  objects. 
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